Goals of this Dagstuhl Seminar

Performance Evaluation is a discipline of Computer Science for some thirty years. It seems time to take stock of what we were doing. That is, provide answers to the following questions: • What are its scientific contributions? • What is its relevance in industry and business? • What is its standing in academia? • Where is the field headed? • What are its success stories and failures? • What are its current burning questions? At this Dagstuhl seminar, we were taking the term workshop literally and have broken up in five working groups. These groups worked in parallel on different areas of Performance Evaluation. Each group prepared a written report during the stay at Dagstuhl. These reports constitute the remainder of this text. The ultimate goal of this seminar is to publish a White Book on Performance Evaluation that will become a catalyst for the advancements of this field as well as a book of reference. Furthermore, each group discussed Success Stories, Failures, Impact of Methods, andBurning Questions of Performance Evaluation. A summary of this discussion is given in the following: Success Stories • Computer architecture, in particular RISC, cache coherence, and multiprogramming. • IEEE 802 suite of protocols, in particular medium access control level. • Data networks, in particular TCP/IP. • Storage systems, in particular RAID and virtual memory. • Operating systems, in particular timesharing, MVS, and scheduling. • Realization and implementation of methodological results in easy-to-use software packages. Failures • Late consideration of performance issues in system design process. • Bad quality of measured input data. • Wrong level of abstraction. • Educational failures – Lack of making known methods to customers. – Lack of training of modeling skills.